The present invention relates generally to the field of electronic imaging, and more particularly, to a method and apparatus for reconstructing images of documents produced using color filter array sensors.
Most electronic color cameras employ a single sensor on which a periodic pattern of dyes of different colors is superimposed. This pattern is known as a color filter array (CFA). Many other color-sampling schemes are possible, such as prisms plus multiple-sensors, random color arrays (as in the human eye), or sampling different colors at different times. However the application of these schemes in consumer digital photography is limited by factors such as the cost of accurate mechanical alignment, fabrication cost, processing complexity, and the degree of motion present in real scenes.
A CFA sensor is used to simultaneously capture three sets of image data in three primary color planes or channels, such as red (R), green (G) and blue (B). A basic CFA sensor is a single two-dimensional array of sensors that are covered by a pattern of red, green and blue filters, such as the Bayer pattern primary CFA illustrated in FIG. 1. The output of a CFA sensor which records only a single color channel at each pixel or senor location is defined herein as a CFA image.
Image processing is used to combine the three primary color planes to create a full-color image from a CFA image. Although the present invention is described while referring to the Bayer pattern CFA, those skilled in the art will appreciate that images obtained from other CFA patterns can be reconstructed using the method and apparatus described herein. An example of another CFA pattern is the Hitachi complementary color (i.e., cyan (C), magenta (M), yellow (Y), and green (G)) CFA, which is illustrated in FIG. 2.
As illustrated in FIG. 3, an approximate full-color full-resolution image stored in memory 306 is generally obtained today using CFA sensors forming part of an image recording module 302 by interpolating the values (e.g., red, green and blue) of the colors that were not sampled at any given pixel in the sensor in interpolator 304. For a Bayer pattern, two forms of interpolation are required: one for synthesizing missing green information, at sites where blue or red are sampled (i.e., luminance reconstruction), and the other for synthesizing missing red/blue information at sites where green or blue/red were sampled (i.e., chrominance reconstruction).
Since CFA sensors like the Bayer pattern sample chrominance less frequently than luminance, there is more risk of generating aliasing artifacts during chrominance reconstruction. Consequently, CFA signal processing has been improved over time to correct such artifacts created during chrominance reconstruction. For more background on CFA imaging and reconstruction see for example the following U.S. patent Nos., which are incorporated herein by reference: U.S. Pat. Nos. 3,971,065; 5,382,976; 5,506,619; 5,629,734.
Signal processing techniques have also been developed for luminance reconstruction. Three common methods for luminance reconstruction are: bilinear interpolation, which takes the average of four surrounding green pixels; low-pass filtering, which finds the convolution of the greens with a finite window approximation of a sinc function at forty five degrees to the rows and columns of the sensor; and median interpolation, which takes the average of the two middle-ranking surrounding green pixels.
Because these known methods of luminance reconstruction introduce artifacts, notably in the vicinity of edges, due to the indiscriminate nature of the averaging that they apply, decision directed interpolation methods have been devised that select an interpolator on the basis of locally detected patterns. Examples of the features that have been employed for deciding which is the appropriate interpolator include: the local-average-thresholded image, and the local gradient and Laplacian values. For additional background on methods aimed at reducing chrominance artifacts see for example the following U.S. patent Nos., which are incorporated herein by reference: U.S. Pat. Nos. 4,642,678; 4,716,455; 4,716,455; 4,724,395; 4,803,548; 5,629,734; 5,373,322; 5,778,106.
Since existing luminance reconstruction methods employ spatial interpolation to reconstruct green channel information, it is inevitable that the effective resolution of CFA reconstructed images containing many small edges, like text in document images, will be lower than images from an equivalently sized gray-scale sensor. This results in greatly reduced legibility of document images for both machines and humans. Accordingly, it would be desirable to provide an electronic camera imaging system that reconstructs missing green channel values in CFA images using other than spatial interpolation to avoid such resolution-loss.